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AESTRACT 

This paper advances a model, called the expected 
opportunity loss model, for curriculum evaluation. This 
decision-making technique utilizes subjective data by ranking courses 
accc rding to their expected contributions to the primary objective of 
the total program. The model also utilizes objective data in the form 
of component costs, and differs from traditional cost-effectiveness 
models in that it places less emphasis on the cost components. The 
purposes of the decision model are (1) to formulate alternatives for 
decision making under uncertainty, and (2) to appraise the probable 
or conditional opportunity loss. The minimum loss is the optimum 
decision. (DE) 
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PROGRAM EVALUATION IN COST BENEFIT TERMS 



Advanced in this paper is an evaluation model applicable to school 
curriculum evaluation. The developing concept is based on some concepts 
of expected and conditional value. Proposed herein is a method for 
determining value judgments of a well-defined educational program. This 
section precedes the expected opportunity loss (EOL) model-a term to be 
defined later. As a technique for decision making, I propose that the 
EOL model is more appropriate for educational program evaluation than the 
traditional cost/effectiveness model because less emphasis is placed on 
cost in the EOL model. 

Data utilised in the EOL decision model are of two types: (1) sub- 

jective data assigned by ranking elements (courses) according to the 
primary objective of the total program and (2) program component costs, 
the objective data base. First, the subjective data base will be 
discussed . 



There is relatively little question that an ordinal scale for 
measuring subjective judgments is possible. However, this model is prim- 
arily dependent on the interval scale ; and the properties of isomorphism, 
equality, inequality, and additivity are necessary to formulate expressed 
rankings or preferences. Utility and subjective value judgment are synony- 
mous in this presentation, where utility has been removed from the classical 
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view involving money. Proposed here is the idea that "utility is a 
function that arithmetizes the relation of preference under conditions 
of risk or uncertainty" (1:47). 

One primary assumption in measurement of the utility of a given 
set of courses is that all materials or program elements are potentially 
quantifiable. Utility measurement of curriculum is, indeed, an obscure 
area in the field of education. However, the approach advanced in this 
paper is concerned with the elusive problem of assigning value judgments 
by rank to educational program elements for the purpose of determining 
order of value/or utility. Program component utility is also defined 
as the relation of preference under uncertainty for the program objec- 
tive achieved per course. The assumption that each program element 
contributes to the value of the total program is of major importance 
in this decision model. 

Ideally, only those individuals experiencing all elements of the 
total program should assign ranks in the evaluation. Furthermore, a sample 
of approximately 30 is recommended to minimize error. In general, the 
possibility exists in a large sample that all elements of a total program 
may be eq lal in rank. If this is the case then the costs, usually unequal, 
become the major contributing elements of choice. 

When participants have ranked all program elements, the probabilities 

of value per program element are determined by transmutation of rank orders 

into units of worth (2:309-336). To calculate the relative percent of 

worth (RPW) per program component, the formula (RPW) = (r. - .5) 100 is used, 

ri 

where r is the assigned rank and n is the total number of program elements 
ranked. The mean relative percent of worth per element is transformed into 







3 



a mean unit of worth (W) per program component. All units of worth, one 
per program component, are subsequently normalized for utilization as 
probabilities of utility in the EOL decision model. 

Subjective values transformed into probabilities of worth form the 
basic foundation for utility measurement in the EOL decision model. An 
assumption supporting scaling judgments or assigning subjective ranks to 
program components is that the participant’s assigned ranks follow the 
normal probability distribution. The goodness-of-f it test is appropriate 
to determine whether the assigned values approximate the normal probability 
distribution. If the subjective values do not approach the normal proba- 
bility distribution, then two alternatives are suggested" (1) A larger 
sample must be obrained, or (2) collapsing cells with less than a given 
number of observed frequencies is a solution. 

Subjective judgments affect the degree of utility, as estimated per 
individual, that results from experiencing each program element. Further- 
more, subjective values affect the original selection of the total program 
objective and courses to be offered. Therefore, the values of both the 
program planners and persons experiencing the program are incorporated in 
the evaluating process. 
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. EXPECTED OPPORTUNITY LOSS 

Introduction 

The purpose of the decision model is to formulate alternatives for 
decision under uncertainty and appraise opportunity loss (3:38-54), In 
general educational terminology, opportunity loss is the aggregate worth 
given up by failing to include the set of courses or program components 
that will produce the greatest benefit based on utility and program 
costs* 

By utilizing subjective and objective data in the evaluation process, 
the educational planner has two primary concerns: (1) To establish what 

the programs have provided for students through the present time and, 

(2) to predict future program effectiveness under analogous conditions. 
Hence, the element of risk is involved in evaluation and planning. Risk 
is primarily dependent on the planners responsible for formulating objec- 
tives, and individuals that estimate values by ranking the elements. The 
following definitions, assumptions, and principles are presented as steps 
in determining alternatives for choice in the EOL decision model. 

Historical Value Judgments 

In the evaluation of the total program it is imperative to know 
something about past value judgments, following the assumption that they 
.y serve as a guide to what may be expected in future programs. Thus, 
the mean unit of worth (W) per program component becomes the historical 
data that is transformed into value probabilities, the sum of which is 
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one. A value probability is defined as that probability which represents 
the d gree of usefulness or utility of a program component. That is, it 
represents the extent of the total program objective reached by a given 
program component. In the EOL model it is impossible for a program element 
to contribute nothing. There will be a probability of worth per element 
that is greater than zero and less than one because e^ch element is assumed 
to contribute some degree of utility to the program. 

To determine the historical value judgments that are based on 
assigned ranks, all mean units of worth (W^) are normalized. For example, 
assume (P n ) program elements and (W^) mean units of worth. The objective 
is to derive the probabilities of value or utility (IL) , where the sum of 
all ( i = 1,2,3 : ••• n ) is one. In Table I the value probability per 
program element is determined as follows: 

V *•' Wi , ( i = 1,2,3, ... n ). 



n 




Program Cost Analysis 

The purpose of the program cost analysis is to specify the selling 
price of the total program. Included in the process of identifying the 
costs of the resources allocated per program element are expenditures 
for administration, instructional salaries and materials, overhead, capital 
outlay, debt service, maintenance, auxiliary services, debt service, fixed 
charges, maintenance and transportation. 

Public education, in theory, is a "break-even" business; therefore, 
for the most elementary budgetary form the selling price and program cost 
must be equal. The EOL model utilizes the "break-even" concept. Costs 
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TABLE I 

HISTORICAL VALUE JUDGMENTS 
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of program co.?r»onents per hour of credit earned per student are standard- 
ized to present the selling price in amenable form for comparison. 

Conditional Worth 

For this decision model , the conditional worth is the difference 

between the cost of one credit hour per program component and the selling 

prices of all elements. Conditional worth (in dollars) is dependent on 

interaction between the program costs and selling prices. The general 

procedure used for calculation of the conditional worth for one credit 

hour of program element (P ) is shown in Table II, The difference between 

i 

the selling prices (S.) of all elements and cost of the given element 

(C^) , (i * 1,2, ,, n) are illustrated in column four. By definition 

the absolute value of S - C is zero. 

i i 

According to the ’'break-even 11 concept, the diagonal of the matrix 

of conditional worth for a total program will have an optimal decision, 

applicable to conditional opportunity loss, of zero. This matrix, 

Table III, is determined by procedures introduced in Table II „ The 

optimal profit decision is: S - C , S -C,...,S -C ,S -C. 

1 12 2 n-1 n-1 n n 

Conditional Opportunity Loss 

Conditional opportunity loss is the difference between the optimal 
profit decision and the conditional worth of the alternative course prop- 
ositions. The optimal profit decision on this model is zero, and conditional 
worth (Tables III and IV) are identical. In the EOL model, the conditional 
opportunity losses are expressed by absolute differences and, by definition, 
cannot be negative. 
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TABLE II 

CONDITIONAL WORTH OF ONE CREDIT 
HOUR OF (P^ - (i = 1,2, ... n ) 



Program 

Elements 



Selling Price of 
Each Element 



Cost of 



(Pi) 



Conditional Worth 

of (Pj) 



C. 

1 



S l- C i 



C i 



s 2 - c. 



S n " C i 
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TABLE III 
CONDITIONAL WORTH 
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TABLE IV 

EXPECTED OPPORTUNITY 
LOSS FOR (P L ) 



Program 

Elements 


Value 

Pi obabilities 


Conditional 

Opportunity 

Loss 


Expected 

Opportunity 

Loss 










p i 


D 1 


s i - c i 


(Sj^ - C t ) 


P 2 


u 2 


s 2 - 


o 2 (S 2 - c.) 










• 


• 


• 


• 


• 


* 


• 


• 


P n 


ff n 


S n " C 1 


D n (8 n - Cj) 

.2, »i («!-«!) 
1 = 1 
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Expected Opportunity Loss 

The EOL for a program element (Pp is the sum of the products 
of each value probability and the conditional opportunity Iosl. . Pro- 
cedures for determining EOL for (Pp are illustrated in Table IV, If 
all costs (Cp and selling price (sp are equal then the conditional 
worth and the conditional opportunity loss is zero. If this is the 
case in Table IV, the EOL is, indeed, zero. Therefore, the solution 
i.i terms of cost is trivial, and the only remaining elements of choice 
are the value probabilities. The minimum EOL per set of alternatives is 
the optimal decision. Table V illustrates the expected loss for a pro- 
gram with (P q ) elements. The expected loss is amenable to predicting the 
success of a future program, assuming like conditions, in terms of 
utility and cost, and is a proposed procedure for evaluating a continuing 
or completed program. 



SUMMARY AND CONCLUSIONS 



The EOL decision model may best be summarized by the flow chart 
in Figure I, Input for the model is basically comprised of program 
utility (U^), program costs (Cp, and program selling price (sp , The 
technique of ranking is utilized to determine utility from a sample of 
students that have experienced the total program, A "break-even 11 prop- 
osition is assumed in the educational institution to determine conditional 
worth (Table III), Therefore, the optimal profit decision is, indeed, 
zero. Those concepts illustrated in Table II, III, and IV represent the 
process whereby the desired output is achieved. The output is the set 
of alternatives, expected losses, illus traced in Table V, The minimum 
jss is the optimal decision. 
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TABLE V 
EXPECTED LOSS 




O 
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FIGURE I 

INFORMATION FLOW FOR THE 
EOL DECIS ION MODEL 
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What are the advantages of EOL over traditional cost/effectiveness 
procedures? One important advantage is the interaction between program 
costs as displayed in Table II. Not only is there interaction between 
program costs in the EOL model, but there is dependence upon all assigned 
value probabilities for each EOL per program (Table IV). There is, in fact, 
double dependence, value and cost, in each program EOL. On the other hand, 
in traditional cost/effectiveness models only a one to one correspondence 
determines the decision ratio. 

This discussion was not intended to imply that a "one-shet" evaluation 
of an educational program is by any means adequate. The two major purposes 
were, in fact, to suggest a working decision model and encourage further 
research and systematic evaluation. When systematic evaluation is con- 
tinued, it is possible to compare the accumulating data and by frequent 
monitoring of the program establish a broad data base for improvement of 
decision-making. This data base involves the EOL criterion, selected 
performance indicators, and subjective judgment of the decision maker. 

All aspects of evaluation should greatly depend on behavioral objectives. 
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